Methods and apparatus to determine engagement levels of audience members

ABSTRACT

Methods and apparatus to determine engagement levels of audience members are disclosed. An example method includes detecting light information in image data representative of an environment including a primary media device; determining whether a secondary media device is in the environment based on the detected light information; and calculating, via a logic circuit, an engagement level for a person with respect to the primary media device based on the secondary media device determination.

FIELD OF THE DISCLOSURE

This patent relates generally to audience measurement and, moreparticularly, to methods and apparatus to determine engagement levels ofaudience members.

BACKGROUND

Audience measurement of media (e.g., broadcast television and/or radio,stored audio and/or video content played back from a memory such as adigital video recorder or a digital video disc, a webpage, audio and/orvideo media presented (e.g., streamed) via the Internet, a video game,etc.) often involves collection of media identifying data (e.g.,signature(s), fingerprint(s), code(s), tuned channel identificationinformation, time of exposure information, etc.) and people data (e.g.,user identifiers, demographic data associated with audience members,etc.). The media identifying data and the people data can be combined togenerate, for example, media exposure data indicative of amount(s)and/or type(s) of people that were exposed to specific piece(s) ofmedia.

In some audience measurement systems, the people data is collected bycapturing a series of images of a media exposure environment (e.g., atelevision room, a family room, a living room, a bar, a restaurant,etc.) and analyzing the images to determine, for example, an identity ofone or more persons present in the media exposure environment, an amountof people present in the media exposure environment during one or moretimes and/or periods of time, etc. The collected people data can becorrelated with media identifying information corresponding to mediadetected as being presented in the media exposure environment to provideexposure data (e.g., ratings data) for that media.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of an example exposure environment includingan example audience measurement device constructed in accordance withthe teachings of this disclosure.

FIG. 2 is a block diagram of an example implementation of the exampleaudience measurement device of FIG. 1.

FIG. 3 is an illustration of an example person tracked by the exampleface detector of FIG. 2.

FIG. 4 is a block diagram of an example implementation of the examplebehavior monitor of FIG. 2.

FIG. 5 is a block diagram of an example implementation of the examplesecondary device detector of FIG. 3.

FIG. 6 is an illustration of an example implementation of a set of lightsignatures of FIG. 5.

FIG. 7 is a flowchart representation of example machine readableinstructions that may be executed to implement the example secondarydevice detector of FIGS. 4 and/or 5.

FIG. 8 is a block diagram of an example processing platform capable ofexecuting the example machine readable instructions of FIG. 7 toimplement the example secondary device detector of FIGS. 4 and/or 5.

DETAILED DESCRIPTION

In some audience measurement systems, people data is collected for amedia exposure environment (e.g., a television room, a family room, aliving room, a bar, a restaurant, an office space, a cafeteria, etc.) bycapturing a series of images of the environment and analyzing the imagesto determine, for example, an identity of one or more persons present inthe media exposure environment, an amount of people present in the mediaexposure environment during one or more times and/or periods of time,etc. The people data can be correlated with media identifyinginformation corresponding to detected media to provide exposure data forthat media. For example, an audience measurement entity (e.g., TheNielsen Company (US), LLC) can calculate ratings for a first piece ofmedia (e.g., a television program) by correlating data collected from aplurality of panelist sites with the demographics of the panelist. Forexample, in each panelist site wherein the first piece of media isdetected in the monitored environment at a first time, media identifyinginformation for the first piece of media is correlated with presenceinformation detected in the environment at the first time. The resultsfrom multiple panelist sites are combined and/or analyzed to provideratings representative of exposure of a population as a whole.

Traditionally, such systems treat each detected person as present forpurposes of calculating the exposure data (e.g., ratings) despite thefact that a first detected person may be paying little or no attentionto the presentation of the media while a second detected person may befocused on (e.g., highly attentive too and/or interacting with) thepresentation of the media.

Example methods, apparatus, and/or articles of manufacture disclosedherein recognize that although a person may be detected as present inthe media exposure environment, the presence of the person does notnecessarily mean that the person is paying attention to (e.g., isengaged with) a certain media presentation device. For example, anaudience measurement device deployed in a living room may detect aperson sitting on a couch in front of a television. According toprevious systems, the person detected in front of the television isdetermined to be engaged with the television and, thus, the media beingpresented by the television. Examples disclosed herein recognize thatalthough the person is sitting in front of the television, the personmay be engaged with a different media device such as, for example, atablet, a laptop computer, a mobile phone, or a desktop computer.Examples disclosed herein recognize that such a person (e.g., a personinteracting with a tablet, a laptop computer, a mobile phone, a desktopcomputer, etc.) is not engaged with the television or at least lessengaged with the television than someone not interacting with adifferent media device (other than the television). For example, theperson may be browsing the Internet on a tablet rather than watching amovie being presented by the television. Alternatively, the person maybe writing a text message on a mobile phone rather than watching atelevision program being presented by the television. Alternatively, theperson may be browsing the Internet on a laptop computer rather thanwatching an on-demand program being presented by the television. In suchinstances, the television is referred to herein as a primary mediadevice and the tablet, mobile phone, and/or laptop computer are referredto herein as secondary media device(s). While the above example refersto a television as a primary media device, examples disclosed herein canbe utilized with additional or alternative types of media presentationdevices serving as the primary media device and/or the secondary mediadevice.

To identify such interactions with secondary media devices, examplesdisclosed herein monitor the environment for light patterns associatedwith a projection of light generated by certain media presentationdevices. In some examples disclosed herein, image data (e.g., a portionof an image corresponding to a detected face of a person) captured ofthe media exposure environment is compared to light signatures known tocorrespond to light signature projected onto a body part of a person(e.g., a face of a person) generated by a display in close proximity(e.g., within three feet) to the person. When examples disclosed hereindetermine that a detected light pattern in the environment resembles oneof the light signatures known to correspond to a projection of lightfrom a secondary media device onto an object (e.g., a face), examplesdisclosed herein determine that the person is (or at least likely is)interacting with the secondary media device (e.g., a tablet, a mobilephone, a laptop computer, a desktop computer, etc.) and, thus, is payinga reduced amount of attention to the primary media device (e.g., atelevision).

Examples disclosed herein also recognize that a mere presence of asecondary media device in a monitored environment may be indicative of areduced engagement with the primary media device. For example, presenceof a laptop computer in an environment including a television as theprimary media device may distract a person from the television (e.g.,due to music, video, and/or images being displayed by the laptopcomputer). Accordingly, in addition to or lieu of the light patterndetection described above, examples disclosed herein detect a glowemanating from secondary media devices such as, for example, a tablet, amobile phone, a laptop computer, a desktop computer, etc. Examplesdisclosed herein determine that a secondary media device is presentbased on such a detected glow. In some instances, to determine whether aperson detected in the environment is interacting with the secondarymedia device associated with the detected glow, examples disclosedherein measure a proximity of the detected to glow to the person.

Examples disclosed herein utilize detections of interactions withsecondary media device(s) and/or presence of secondary media device(s)to measure attentiveness of the audience member(s) with respect to theprimary media device. An example measure of attentiveness for anaudience member provided by examples disclosed herein is referred toherein as an engagement level. In some examples disclosed herein,individual engagement levels of separate audience members (who may bephysically located at a same specific exposure environment and/or atmultiple different exposure environments) are combined, aggregated,statistically adjusted, and/or extrapolated to formulate a collectiveengagement level for an audience at one or more physical locations. Insome examples, a person specific engagement level for each audiencemember with respect to particular media is calculated in real time(e.g., virtually simultaneously with) as the primary media devicepresents the particular media.

Additionally or alternatively, examples disclosed herein use theanalysis of the light patterns detected in the monitored environment toidentify a type of the secondary media device being used. For example,depending on which of the known light signatures matches the detectedlight pattern in the environment, examples disclosed herein may identifythe type of the secondary media device as a tablet or a mobiletelephone. Additionally or alternatively, examples disclosed herein usethe analysis of the light patterns detected in the monitored environmentto identify the secondary media device being used by, for example, brandname, model number, generation, etc. For example, depending on which oneof the known light signatures matches the detected light pattern in theenvironment, examples disclosed herein may identify the secondary mediadevice as corresponding to a particular manufacturer (e.g., an Apple®product) or even as corresponding to a particular product (e.g., anApple® iPhone®, an Apple® iPhone5®, a Samsung® product, a Samsung®Galaxy S3®, etc.)

FIG. 1 is an illustration of an example media exposure environment 100including an information presentation device 102, a multimodal sensor104, and a meter 106 for collecting audience measurement data. In theillustrated example of FIG. 1, the media exposure environment 100 is aroom of a household (e.g., a room in a home of a panelist such as thehome of a “Nielsen family”) that has been statistically selected todevelop television ratings data for a population/demographic ofinterest. In the illustrated example, one or more persons of thehousehold have registered with an audience measurement entity (e.g., byagreeing to be a panelist) and have provided their demographicinformation to the audience measurement entity as part of a registrationprocess to enable associating demographics with viewing activities(e.g., media exposure).

In some examples, the audience measurement entity provides themultimodal sensor 104 to the household. In some examples, the multimodalsensor 104 is a component of a media presentation system purchased bythe household such as, for example, a camera of a video game system 108(e.g., Microsoft® Kinect®) and/or piece(s) of equipment associated witha video game system (e.g., a Kinect® sensor). In such examples, themultimodal sensor 104 may be repurposed and/or data collected by themultimodal sensor 104 may be repurposed for audience measurement.

In the illustrated example of FIG. 1, the multimodal sensor 104 isplaced above the information presentation device 102 at a position forcapturing image and/or audio data of the environment 100. In someexamples, the multimodal sensor 104 is positioned beneath or to a sideof the information presentation device 102 (e.g., a television or otherdisplay). In the illustrated example of FIG. 1, the example informationpresentation device 102 is referred to as a primary media device becausethe multimodal sensor 104 is configured to monitor the environment 100relative to the information presentation device 102. The exampleenvironment 100 of FIG. 1 also includes a first secondary media device112 with which an audience member 110 is interacting and a secondsecondary media device 114 resting on a table. As described below, theexample meter 106 of FIG. 1 is capable of (1) detecting usage of thefirst secondary media device 112 by the audience member 110, (2)identifying a type of the first secondary media device 112 with whichthe audience member is interacting (e.g., a tablet, phone, etc.), (3)identifying the first secondary media device 112 itself (e.g.,manufacturer, model, etc.), (4) detecting a glow associated with thesecond secondary media device 114, and/or (5) factoring the detectedinteraction, type, and/or identity of the first secondary media device112 and/or the presence of the second secondary media device 114 into anengagement level calculation for the audience member 110 with respect tothe primary media device 102. In other words, the example meter 106 ofFIG. 1 determines whether the audience member 110 is interacting withand/or is likely to be interacting with the first secondary media device112 and/or the second secondary media device 114 and considers suchdeterminations when measuring a level of engagement of the audiencemember 110 with the primary media device 102. In some examples, themeter 106 of FIG. 1 uses the detected interaction with the secondarymedia device(s) 112, 114 to increase or decrease an already calculatedengagement level for the audience member 110. In some examples, theexample meter 106 of FIG. 1 calculates an engagement level of theaudience member 110 with the primary media device 102 based solely onthe detected (or not detected) interaction with the secondary mediadevice(s) 112, 114. The example detection of secondary media deviceusage and the engagement calculations disclosed herein are described indetail below in connection with FIGS. 5-7.

In some examples, the multimodal sensor 104 is integrated with the videogame system 108. For example, the multimodal sensor 104 may collectimage data (e.g., three-dimensional data and/or two-dimensional data)using one or more sensors for use with the video game system 108 and/ormay also collect such image data for use by the meter 106. In someexamples, the multimodal sensor 104 employs a first type of image sensor(e.g., a two-dimensional sensor) to obtain image data of a first type(e.g., two-dimensional data) and collects a second type of image data(e.g., three-dimensional data) from a second type of image sensor (e.g.,a three-dimensional sensor). In some examples, only one type of sensoris provided by the video game system 108 and a second sensor is added bythe audience measurement system.

In the example of FIG. 1, the meter 106 is a software meter provided forcollecting and/or analyzing the data from, for example, the multimodalsensor 104 and other media identification data collected as explainedbelow. In some examples, the meter 106 is installed in the video gamesystem 108 (e.g., by being downloaded to the same from a network, bybeing installed at the time of manufacture, by being installed via aport (e.g., a universal serial bus (USB) from a jump drive provided bythe audience measurement company, by being installed from a storage disc(e.g., an optical disc such as a BluRay disc, Digital Versatile Disc(DVD) or CD (compact Disk), or by some other installation approach).Executing the meter 106 on the panelist's equipment is advantageous inthat it reduces the costs of installation by relieving the audiencemeasurement entity of the need to supply hardware to the monitoredhousehold). In other examples, rather than installing the software meter106 on the panelist's consumer electronics, the meter 106 is a dedicatedaudience measurement unit provided by the audience measurement entity.In such examples, the meter 106 may include its own housing, processor,memory and software to perform the desired audience measurementfunctions. In such examples, the meter 106 is adapted to communicatewith the multimodal sensor 104 via a wired or wireless connection. Insome such examples, the communications are affected via the panelist'sconsumer electronics (e.g., via a video game console). In other example,the multimodal sensor 104 is dedicated to audience measurement and,thus, no interaction with the consumer electronics owned by the panelistis involved.

The example audience measurement system of FIG. 1 can be implemented inadditional and/or alternative types of environments such as, forexample, a room in a non-statistically selected household, a theater, arestaurant, a tavern, a retail location, an arena, etc. For example, theenvironment may not be associated with a panelist of an audiencemeasurement study, but instead may simply be an environment associatedwith a purchased XBOX® and/or Kinect® system. In some examples, theexample audience measurement system of FIG. 1 is implemented, at leastin part, in connection with additional and/or alternative types of mediapresentation devices such as, for example, a radio, a computer, atablet, a cellular telephone, and/or any other communication device ableto present media to one or more individuals.

In the illustrated example of FIG. 1, the primary media device 102(e.g., a television) is coupled to a set-top box (STB) that implements adigital video recorder (DVR) and a digital versatile disc (DVD) player.Alternatively, the DVR and/or DVD player may be separate from the STB.In some examples, the meter 106 of FIG. 1 is installed (e.g., downloadedto and executed on) and/or otherwise integrated with the STB. Moreover,the example meter 106 of FIG. 1 can be implemented in connection withadditional and/or alternative types of media presentation devices suchas, for example, a radio, a computer monitor, a video game consoleand/or any other communication device able to present content to one ormore individuals via any past, present or future device(s), medium(s),and/or protocol(s) (e.g., broadcast television, analog television,digital television, satellite broadcast, Internet, cable, etc.).

As described in detail below, the example meter 106 of FIG. 1 utilizesthe multimodal sensor 104 to capture a plurality of time stamped framesof image data, depth data, and/or audio data from the environment 100.In example of FIG. 1, the multimodal sensor 104 of FIG. 1 is part of thevideo game system 108 (e.g., Microsoft® XBOX®, Microsoft® Kinect®).However, the example multimodal sensor 104 can be associated and/orintegrated with the STB, associated and/or integrated with the primarymedia device 102, associated and/or integrated with a BlueRay® playerlocated in the environment 100, or can be a standalone device (e.g., aKinect® sensor bar, a dedicated audience measurement meter, etc.),and/or otherwise implemented. In some examples, the meter 106 isintegrated in the STB or is a separate standalone device and themultimodal sensor 104 is the Kinect® sensor or another sensing device.The example multimodal sensor 104 of FIG. 1 captures images within afixed and/or dynamic field of view. To capture depth data, the examplemultimodal sensor 104 of FIG. 1 uses a laser or a laser array to projecta dot pattern onto the environment 100. Depth data collected by themultimodal sensor 104 can be interpreted and/or processed based on thedot pattern and how the dot pattern lays onto objects of the environment100. In the illustrated example of FIG. 1, the multimodal sensor 104also captures two-dimensional image data via one or more cameras (e.g.,infrared sensors) capturing images of the environment 100. In theillustrated example of FIG. 1, the multimodal sensor 104 also capturesaudio data via, for example, a directional microphone. As described ingreater detail below, the example multimodal sensor 104 of FIG. 1 iscapable of detecting some or all of eye position(s) and/or movement(s),skeletal profile(s), pose(s), posture(s), body position(s), personidentit(ies), body type(s), etc. of the individual audience members. Insome examples, the data detected via the multimodal sensor 104 is usedto, for example, detect and/or react to a gesture, action, or movementtaken by the corresponding audience member. The example multimodalsensor 104 of FIG. 1 is described in greater detail below in connectionwith FIG. 2.

As described in detail below in connection with FIG. 2, the examplemeter 106 of FIG. 1 monitors the environment 100 to identify media beingpresented (e.g., displayed, played, etc.) by the primary media device102 and/or other media presentation devices to which the audience isexposed (e.g., the secondary media device(s) 112, 114). In someexamples, identification(s) of media to which the audience is exposedare correlated with the presence information collected by the multimodalsensor 104 to generate exposure data for the media. In some examples,identification(s) of media to which the audience is exposed arecorrelated with behavior data (e.g., engagement levels) collected by themultimodal sensor 104 to additionally or alternatively generateengagement ratings for the media presented by, for example, the primarymedia device 102.

FIG. 2 is a block diagram of an example implementation of the examplemeter 106 of FIG. 1. The example meter 106 of FIG. 2 includes anaudience detector 200 to develop audience composition informationregarding, for example, the audience member 110 of FIG. 1. The examplemeter 106 of FIG. 2 also includes a media detector 202 to collect mediainformation regarding, for example, media presented in the environment100 of FIG. 1. The example multimodal sensor 104 of FIG. 2 includes athree-dimensional sensor and a two-dimensional sensor. The example meter106 may additionally or alternatively receive three-dimensional dataand/or two-dimensional data representative of the environment 100 fromdifferent source(s). For example, the meter 106 may receivethree-dimensional data from the multimodal sensor 104 andtwo-dimensional data from a different component. Alternatively, themeter 106 may receive two-dimensional data from the multimodal sensor104 and three-dimensional data from a different component.

In some examples, to capture three-dimensional data, the multimodalsensor 104 projects an array or grid of dots (e.g., via one or morelasers) onto objects of the environment 100. The dots of the arrayprojected by the example multimodal sensor 104 have respective x-axiscoordinates and y-axis coordinates and/or some derivation thereof. Theexample multimodal sensor 104 of FIG. 2 uses feedback received inconnection with the dot array to calculate depth values associated withdifferent dots projected onto the environment 100. Thus, the examplemultimodal sensor 104 generates a plurality of data points. Each suchdata point has a first component representative of an x-axis position inthe environment 100, a second component representative of a y-axisposition in the environment 100, and a third component representative ofa z-axis position in the environment 100. As used herein, the x-axisposition of an object is referred to as a horizontal position, they-axis position of the object is referred to as a vertical position, andthe z-axis position of the object is referred to as a depth positionrelative to the multimodal sensor 104. The example multimodal sensor 104of FIG. 2 may utilize additional or alternative type(s) ofthree-dimensional sensor(s) to capture three-dimensional datarepresentative of the environment 100.

While the example multimodal sensor 104 implements a laser to projectsthe plurality grid points onto the environment 100 to capturethree-dimensional data, the example multimodal sensor 104 of FIG. 2 alsoimplements an image capturing device, such as a camera, that capturestwo-dimensional image data representative of the environment 100. Insome examples, the image capturing device includes an infrared imagerand/or a charge coupled device (CCD) camera. In some examples, themultimodal sensor 104 only captures data when the primary media device102 is in an “on” state and/or when the media detector 202 determinesthat media is being presented in the environment 100 of FIG. 1. Theexample multimodal sensor 104 of FIG. 2 may also include one or moreadditional sensors to capture additional or alternative types of dataassociated with the environment 100.

Further, the example multimodal sensor 104 of FIG. 2 includes adirectional microphone array capable of detecting audio in certainpatterns or directions in the media exposure environment 100. In someexamples, the multimodal sensor 104 is implemented at least in part by aMicrosoft® Kinect® sensor.

The example audience detector 200 of FIG. 2 includes an ambient lightcondition sensor 204 to identify a lighting condition associated withthe example environment 100. The example ambient light condition sensor204 is implemented by, for example, one or more photo cells capable ofdetecting an amount of light present in the environment 100 and/or otherlight-based characteristics of the environment 100. In some examples,the ambient light condition sensor 204 of FIG. 2 additionally oralternatively implements a timer to determine a time of day. The exampleambient light condition sensor 204 uses the determined time of day to,for example, attribute the detected amount of light to daylight and/orartificial light (e.g., light from a lamp). Additionally oralternatively, the example ambient light condition sensor 204 implementsa first sensor to detect an amount of natural light (e.g., daylight) anda second sensor to detect an amount of artificial light (e.g., lightgenerated by a light bulb). As described in greater detail below inconnection with FIG. 5, the lighting characteristics of the environment100 captured by the ambient light condition sensor 204 are used toselect one of a plurality of sets of light signatures for an analysis ofthe environment 100. Different ones of the light signatures correspondto different lighting conditions and, thus, the ambient light conditionsensor 204 enables selection of the appropriate set of light signaturesfor the analysis of the environment 100.

The example audience detector 200 of FIG. 2 includes a people analyzer206, a behavior monitor 208, a time stamper 210, and a memory 212. Inthe illustrated example of FIG. 2, data obtained by the multimodalsensor 104 of FIG. 2, such as depth data, two-dimensional image data,and/or audio data is conveyed to the people analyzer 206. The examplepeople analyzer 206 of FIG. 2 generates a people count or tallyrepresentative of a number of people in the environment 100 for a frameof captured image data. The rate at which the example people analyzer206 generates people counts is configurable. In the illustrated exampleof FIG. 2, the example people analyzer 206 instructs the examplemultimodal sensor 104 to capture data (e.g., three-dimensional and/ortwo-dimensional data) representative of the environment 100 every fiveseconds. However, the example people analyzer 206 can receive and/oranalyze data at any suitable rate.

The example people analyzer 206 of FIG. 2 determines how many peopleappear in a frame in any suitable manner using any suitable technique.For example, the people analyzer 206 of FIG. 2 recognizes a generalshape of a human body and/or a human body part, such as a head and/ortorso. Additionally or alternatively, the example people analyzer 206 ofFIG. 2 may count a number of “blobs” that appear in the frame and counteach distinct blob as a person. Recognizing human shapes and counting“blobs” are illustrative examples and the people analyzer 206 of FIG. 2can count people using any number of additional and/or alternativetechniques. An example manner of counting people is described byRamaswamy et al. in U.S. patent application Ser. No. 10/538,483, filedon Dec. 11, 2002, now U.S. Pat. No. 7,203,338, which is herebyincorporated herein by reference in its entirety.

In the illustrated example of FIG. 2, the people analyzer 206 tracks aposition of each detected person in the environment 100 of FIG. 1. Inparticular, the example people analyzer 206 of FIG. 2 generates acoordinate (e.g., an X-Y coordinate or an X-Y-Z coordinate) for eachdetected person. FIG. 3 illustrates a detected person 300 and acoordinate 302 generated by the example people analyzer 206 to track aposition of the person 300. The example person 300 of FIG. 3 maycorrespond to the audience member 110 of FIG. 1. In some examples, theexample coordinate 302 of FIG. 3 and/or any other suitable positiontracking data generated by, for example, the people analyzer 206 isutilized by the behavior monitor 208. For example, as described below,the example behavior monitor 208 uses the coordinate 302 of FIG. 3 tofocus an analysis of image data on an area of the environment 100 knownto include the person 300 (as identified by the people analyzer 206).

Additionally, the example people analyzer 206 of FIG. 2 executes afacial recognition procedure such that people captured in the frames canbe individually identified. In some examples, the audience detector 200may have additional or alternative methods and/or components to identifypeople in the frames. For example, the audience detector 200 of FIG. 2can implement a feedback system to which the members of the audienceprovide (e.g., actively and/or passively) identification to the meter106. To identify people in the frames, the example people analyzer 206includes or has access to a collection (e.g., stored in a database) offacial signatures (e.g., image vectors). Each facial signature of theillustrated example corresponds to a person having a known identity tothe people analyzer 206. The collection includes an identifier (ID) foreach known facial signature that corresponds to a known person. Forexample, in reference to FIG. 1, the collection of facial signatures maycorrespond to frequent visitors and/or members of the householdassociated with the room 100. The example people analyzer 206 of FIG. 2analyzes one or more regions of a frame thought to correspond to a humanface and develops a pattern or map for the region(s) (e.g., using thedepth data provided by the multimodal sensor 104). The pattern or map ofthe region represents a facial signature of the detected human face. Insome examples, the pattern or map is mathematically represented by oneor more vectors. The example people analyzer 206 of FIG. 2 compares thedetected facial signature to entries of the facial signature collection.When a match is found, the example people analyzer 206 has successfullyidentified at least one person in the frame. In such instances, theexample people analyzer 206 of FIG. 2 records (e.g., in a memory addressaccessible to the people analyzer 206) the ID associated with thematching facial signature of the collection. When a match is not found,the example people analyzer 206 of FIG. 2 retries the comparison orprompts the audience for information that can be added to the collectionof known facial signatures for the unmatched face. More than onesignature may correspond to the same face (i.e., the face of the sameperson). For example, a person may have one facial signature whenwearing glasses and another when not wearing glasses. A person may haveone facial signature with a beard, and another when cleanly shaven.

Each entry of the collection of known people used by the example peopleanalyzer 206 of FIG. 2 also includes a type for the corresponding knownperson. For example, the entries of the collection may indicate that afirst known person is a child of a certain age and/or age range and thata second known person is an adult of a certain age and/or age range. Ininstances in which the example people analyzer 206 of FIG. 2 is unableto determine a specific identity of a detected person, the examplepeople analyzer 206 of FIG. 2 estimates a type for the unrecognizedperson(s) detected in the exposure environment 100. For example, thepeople analyzer 206 of FIG. 2 estimates that a first unrecognized personis a child, that a second unrecognized person is an adult, and that athird unrecognized person is a teenager. The example people analyzer 206of FIG. 2 bases these estimations on any suitable factor(s) such as, forexample, height, head size, body proportion(s), etc.

In the illustrated example, data obtained by the multimodal sensor 104of FIG. 2 is conveyed to the behavior monitor 208. As described indetail below in connection with FIGS. 4-7, the data conveyed to theexample behavior monitor 208 of FIG. 2 is used by examples disclosedherein to identify behavior(s) and/or generate engagement level(s) forpeople appearing in the environment 100 with respect to, for example,the primary media device 102. For example, the image data captured bythe multimodal sensor 104 is analyzed to determine whether a lightsignature known to correspond to use of a secondary media device (e.g.,the secondary media device 112 of FIG. 1) appears in the image data.That is, the example behavior monitor 208 of FIG. 2 determines whetherthe audience member 110 is interacting with the secondary media device112 based on the detected light pattern, thereby indicatingdisengagement (e.g., no attention or a reduced amount of attention paid)from the primary media device 102 of FIG. 1. In some examples, theexample behavior monitor 208 of FIG. 2 uses the detection of theinteraction with the secondary media device 112 to calculate anengagement level for the audience member 110 with respect to the primarymedia device 102. In some examples, if the detected light patterncorresponds to a light signature known to correspond to a particularmedia device (e.g., a particular brand and/or model of device) and/or aparticular type of media device (e.g., a tablet, a mobile telephone, alaptop computer, a desktop computer, etc.), the example behavior monitor208 of FIG. 2 determines that the particular media device and/or theparticular type of media device is being used in the environment 100. Insome examples, the behavior monitor 208 of FIG. 2 determines whether asecondary media device (e.g., the first and/or second secondary mediadevices 112, 114) are present in the environment based on the detectedlight pattern and/or based a detected glow emanating from the secondarymedia device(s). The example behavior monitor 208 is described in detailbelow in connection with FIGS. 4-7.

The example people analyzer 206 of FIG. 2 outputs the calculatedtallies, identification information, person type estimations forunrecognized person(s), and/or corresponding image frames to the timestamper 210. Similarly, the example behavior monitor 208 outputs data(e.g., calculated behavior(s), engagement level(s), media selection(s),media device identifier(s), etc.) to the time stamper 210. The timestamper 210 of the illustrated example includes a clock and a calendar.The example time stamper 210 associates a time period (e.g., 1:00 a.m.Central Standard Time (CST) to 1:01 a.m. CST) and date (e.g., Jan. 1,2012) with each calculated people count, identifier, frame, behavior,engagement level, media selection, etc., by, for example, appending theperiod of time and data information to an end of the data. A datapackage (e.g., the people count(s), the time stamp(s), the mediaidentifier(s), the date and time, the engagement level(s), the behaviordata, the media device identifier(s), the image data, etc.) is stored inthe memory 212.

The memory 212 may include a volatile memory (e.g., Synchronous DynamicRandom Access Memory (SDRAM), Dynamic Random Access Memory (DRAM),RAMBUS Dynamic Random Access Memory (RDRAM, etc.) and/or a non-volatilememory (e.g., flash memory). The memory 212 may include one or moredouble data rate (DDR) memories, such as DDR, DDR2, DDR3, mobile DDR(mDDR), etc. The memory 212 may additionally or alternatively includeone or more mass storage devices such as, for example, hard drivedisk(s), compact disk drive(s), digital versatile disk drive(s), etc.When the example meter 106 is integrated into, for example the videogame system 108 of FIG. 1, the meter 106 may utilize memory of the videogame system 108 to store information such as, for example, the peoplecounts, the image data, the engagement levels, etc.

The example time stamper 210 of FIG. 2 also receives data from theexample media detector 202. The example media detector 202 of FIG. 2detects presentation(s) of media in the media exposure environment 100and/or collects identification information associated with the detectedpresentation(s). For example, the media detector 202, which may be inwired and/or wireless communication with the presentation device (e.g.,television) 102, the multimodal sensor 104, the video game system 108,the STB 110, and/or any other component(s) of FIG. 1, can identify apresentation time and a source of a presentation. The presentation timeand the source identification data may be utilized to identify theprogram by, for example, cross-referencing a program guide configured,for example, as a look up table. In such instances, the sourceidentification data may be, for example, the identity of a channel(e.g., obtained by monitoring a tuner of the STB of FIG. 1 or a digitalselection made via a remote control signal) currently being presented onthe information presentation device 102.

Additionally or alternatively, the example media detector 202 canidentify the presentation by detecting codes (e.g., watermarks) embeddedwith or otherwise conveyed (e.g., broadcast) with media being presentedvia the STB and/or the primary media device 102. As used herein, a codeis an identifier that is transmitted with the media for the purpose ofidentifying and/or for tuning to (e.g., via a packet identifier headerand/or other data used to tune or select packets in a multiplexed streamof packets) the corresponding media. Codes may be carried in the audio,in the video, in metadata, in a vertical blanking interval, in a programguide, in content data, or in any other portion of the media and/or thesignal carrying the media. In the illustrated example, the mediadetector 202 extracts the codes from the media. In some examples, themedia detector 202 may collect samples of the media and export thesamples to a remote site for detection of the code(s).

Additionally or alternatively, the media detector 202 can collect asignature representative of a portion of the media. As used herein, asignature is a representation of some characteristic of signal(s)carrying or representing one or more aspects of the media (e.g., afrequency spectrum of an audio signal). Signatures may be thought of asfingerprints of the media. Collected signature(s) can be comparedagainst a collection of reference signatures of known media to identifythe tuned media. In some examples, the signature(s) are generated by themedia detector 202. Additionally or alternatively, the media detector202 may collect samples of the media and export the samples to a remotesite for generation of the signature(s). In the example of FIG. 2,irrespective of the manner in which the media of the presentation isidentified (e.g., based on tuning data, metadata, codes, watermarks,and/or signatures), the media identification information is time stampedby the time stamper 210 and stored in the memory 212.

In the illustrated example of FIG. 2, the output device 214 periodicallyand/or aperiodically exports data (e.g., media identificationinformation, audience identification information, etc.) from the memory214 to a data collection facility 216 via a network (e.g., a local-areanetwork, a wide-area network, a metropolitan-area network, the Internet,a digital subscriber line (DSL) network, a cable network, a power linenetwork, a wireless communication network, a wireless mobile phonenetwork, a Wi-Fi network, etc.). In some examples, the example meter 106utilizes the communication abilities (e.g., network connections) of thevideo game system 108 to convey information to, for example, the datacollection facility 216. In the illustrated example of FIG. 2, the datacollection facility 216 is managed and/or owned by an audiencemeasurement entity (e.g., The Nielsen Company (US), LLC). The audiencemeasurement entity associated with the example data collection facility216 of FIG. 2 utilizes the people tallies generated by the peopleanalyzer 206 and/or the personal identifiers generated by the peopleanalyzer 206 in conjunction with the media identifying data collected bythe media detector 202 to generate exposure information. The informationfrom many panelist locations may be compiled and analyzed to generateratings representative of media exposure by one or more populations ofinterest.

In some examples, the data collection facility 216 employs analyzes thebehavior/engagement level information generated by the example behaviormonitor 208 to, for example, generate engagement level ratings for mediaidentified by the media detector 202. In some examples, the engagementlevel ratings are used to determine whether a retroactive fee is due toa service provider from an advertiser due to a certain engagement levelexisting at a time of presentation of content of the advertiser.

Alternatively, analysis of the data (e.g., data generated by the peopleanalyzer 206, the behavior monitor 208, and/or the media detector 202)may be performed locally (e.g., by the example meter 106 of FIG. 2) andexported via a network or the like to a data collection facility (e.g.,the example data collection facility 216 of FIG. 2) for furtherprocessing. For example, the amount of people (e.g., as counted by theexample people analyzer 206) and/or engagement level(s) (e.g., ascalculated by the example behavior monitor 208) in the exposureenvironment 100 at a time (e.g., as indicated by the time stamper 210)in which a sporting event (e.g., as identified by the media detector202) was presented by the primary media device 102 can be used in aexposure calculation and/or engagement calculation for the sportingevent. In some examples, additional information (e.g., demographic dataassociated with one or more people identified by the people analyzer206, geographic data, etc.) is correlated with the exposure informationand/or the engagement information by the audience measurement entityassociated with the data collection facility 216 to expand theusefulness of the data collected by the example meter 106 of FIGS. 1and/or 2. The example data collection facility 216 of the illustratedexample compiles data from a plurality of monitored exposureenvironments (e.g., other households, sports arenas, bars, restaurants,amusement parks, transportation environments, retail locations, etc.)and analyzes the data to generate exposure ratings and/or engagementratings for geographic areas and/or demographic sets of interest.

While an example manner of implementing the meter 106 of FIG. 1 has beenillustrated in FIG. 2, one or more of the elements, processes and/ordevices illustrated in FIG. 2 may be combined, divided, re-arranged,omitted, eliminated and/or implemented in any other way. Further, theexample audience detector 200, the example media detector 202, theexample people analyzer 206, the example behavior monitor 208, theexample time stamper 210, the example output device 214 and/or, moregenerally, the example meter 106 of FIG. 2 may be implemented byhardware, software, firmware and/or any combination of hardware,software and/or firmware. Thus, for example, any of the example audiencedetector 200, the example media detector 202, the example peopleanalyzer 206, the behavior monitor 208, the example time stamper 210,the example output device 214 and/or, more generally, the example meter106 of FIG. 2 could be implemented by one or more circuit(s),programmable processor(s), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)), etc. When any of the apparatusor system claims of this patent are read to cover a purely softwareand/or firmware implementation, at least one of the example audiencedetector 200, the example media detector 202, the example peopleanalyzer 206, the behavior monitor 208, the example time stamper 210,the example output device 214 and/or, more generally, the example meter106 of FIG. 2 are hereby expressly defined to include a tangiblecomputer readable storage medium such as a storage device (e.g., memory)or an optical storage disc (e.g., a DVD, a CD, a Bluray disc) storingthe software and/or firmware. Further still, the example meter 106 ofFIG. 2 may include one or more elements, processes and/or devices inaddition to, or instead of, those illustrated in FIG. 2, and/or mayinclude more than one of any or all of the illustrated elements,processes and devices.

FIG. 4 is a block diagram of an example implementation of the examplebehavior monitor 208 of FIG. 2. As described above in connection withFIG. 2, the example behavior monitor 208 of FIG. 4 receives data fromthe multimodal sensor 104 and coordinate information associated with adetected person from the example people analyzer 206 of FIG. 2. Theexample behavior monitor 208 of FIG. 4 processes and/or interprets thedata provided by the multimodal sensor 104 and/or the people analyzer206 to detect one or more aspects of behavior exhibited by, for example,the audience member 110 of FIG. 1 including, for example, interactionswith secondary media devices, such as the secondary media device 112 ofFIG. 1. In particular, the example behavior monitor 208 of FIG. 4includes an engagement level calculator 400 that uses indications ofcertain behaviors and/or interactions detected via the multimodal sensor104 to generate an attentiveness metric (e.g., engagement level) foreach detected audience member with respect to the primary media device102 of FIG. 1. In the illustrated example, the engagement levelcalculated by the engagement level calculator 400 is indicative of howattentive the respective audience member is to the primary media device102 of FIG. 1. While described herein as calculating engagement levelsfor the primary media device 102 of FIG. 2, the example behavior monitor208 of FIG. 4 is also capable of generating engagement level(s) for theenvironment 100 as a whole and/or for media presentation device(s) otherthan the primary media device 102 (e.g., for a secondary media device).For example, when the engagement level calculator 400 determines thatthe audience member 110 of FIG. 1 is sleeping, the engagement level forany media presentation device of the environment 100 is affected by thedetection (e.g., is set to a level indicative of disengagement from therespective media presentation device).

The metric generated by the example engagement level calculator 400 ofFIG. 4 is any suitable type of value such as, for example, a numericscore based on a scale, a percentage, a categorization, one of aplurality of levels defined by respective thresholds, etc. In someexamples, the engagement metric is generated by referencing one or morelookup tables each having, for example, a plurality of threshold valuesand corresponding scores.

In some examples, the metric generated by the example engagement levelcalculator 400 of FIG. 4 is an aggregate score or percentage (e.g., aweighted average) formed by combining a plurality of individualengagement level scores or percentages based on different data and/ordetections (e.g., to form one or more collective engagement levels). Forexample, as described below, the example engagement level calculator 400of FIG. 4 includes a plurality of different components each capable ofgenerating a measurement of engagement for the audience member 110. Insome instances, the example engagement level calculator 400 of FIG. 4combines two or more of the separately generated engagement measurementsto form an aggregate engagement score for the audience member.

In the illustrated example of FIG. 4, the engagement level calculator400 includes an eye tracker 402 to utilize eye position and/or movementdata provided by the multimodal sensor 104. In some examples, the eyetracker 402 utilizes coordinate information (e.g., the coordinate 302 ofFIG. 3) provided by the example people analyzer 206 to focus an analysison a particular portion of the data provided by the multimodal sensor104 known to include (or at least previously include) a person. Theexample eye tracker 402 of FIG. 4 uses the eye position and/or movementdata to determine or estimate whether, for example, a detected audiencemember is looking in a direction of the primary media device 102,whether the audience member is looking away from the primary mediadevice 102, whether the audience member is looking in the generalvicinity of the primary media device 102, or otherwise engaged ordisengaged from the primary media device 102. That is, the example eyetracker 402 of FIG. 4 categorizes how closely a gaze of the detectedaudience member is to the primary media device 102 based on, forexample, an angular difference (e.g., an angle of a certain degree)between a direction of the detected gaze and a direct line of sightbetween the audience member and the primary media device 102. Theexample eye tracker 402 of FIG. 4 determines a direct line of sightbetween a first member of the audience and the primary media device 102.Further, the example eye tracker 402 of FIG. 4 determines a current gazedirection of the first audience member. The example eye tracker 402 ofFIG. 4 calculates the angular difference between the direct line ofsight and the current gaze direction. In some examples the eye tracker402 of FIG. 4 calculates a plurality of angles between a first vectorrepresentative of the direct line of sight and a second vectorrepresentative of the gaze direction. In such instances, the example eyetracker 402 of FIG. 4 includes more than one dimension in thecalculation of the difference between the direct line of sight and thegaze direction.

In some examples, the eye tracker 402 of FIG. 4 calculates a likelihoodthat the respective audience member is looking at the primary mediadevice 102 based on, for example, the calculated difference between thedirect line of sight and the gaze direction. For example, the eyetracker 402 of FIG. 4 compares the calculated difference to one or morethresholds to select one of a plurality of categories (e.g., lookingaway, looking in the general vicinity of the primary media device 102,looking directly at the primary media device 102, etc.). In someexamples, the eye tracker 402 of FIG. 4 translates the calculateddifference (e.g., degrees) between the direct line of sight and the gazedirection into a numerical representation of a likelihood of engagement.For example, the eye tracker 402 of FIG. 4 determines a percentageindicative of a likelihood that the audience member is engaged with theprimary media device 102 and/or indicative of a level of engagement ofthe audience member with the primary media device 102. In suchinstances, higher percentages indicate proportionally higher levels ofattention or engagement.

In some examples, the example eye tracker 402 of FIG. 4 combinesmeasurements and/or calculations taken in connection with a plurality offrames (e.g., consecutive frames). For example, the likelihoods ofengagement calculated by the example eye tracker 402 of FIG. 4 can becombined (e.g., averaged) for a period of time spanning the plurality offrames to generate a collective likelihood that the audience memberlooked at the primary media device 102 for the period of time. In someexamples, the likelihoods calculated by the example eye tracker 402 ofFIG. 4 are translated into respective percentages indicative of howlikely the corresponding audience member(s) are looking at the primarymedia device 102 over the corresponding period(s) of time. Additionallyor alternatively, the example eye tracker 402 of FIG. 4 combinesconsecutive periods of time and the respective likelihoods to determinewhether the audience member(s) were looking at the primary media device102 through consecutive frames. Detecting that the audience member(s)likely viewed the presentation device 102 through multiple consecutiveframes may indicate a higher level of engagement with the television, asopposed to indications that the audience member frequently switched fromlooking at the presentation device 102 and looking away from thepresentation device 102. For example, the eye tracker 402 of FIG. 4 maycalculate a percentage (e.g., based on the angular difference detectiondescribed above) representative of a likelihood of engagement for eachof twenty consecutive frames. In some examples, the eye tracker 402 ofFIG. 4 calculates an average of the twenty percentages and compares theaverage to one or more thresholds, each indicative of a level ofengagement. Depending on the comparison of the average to the one ormore thresholds, the example eye tracker 402 of FIG. 4 determines alikelihood or categorization of the level of engagement of thecorresponding audience member for the period of time corresponding tothe twenty frames.

In the illustrated example of FIG. 4, the engagement calculator 400includes a pose identifier 404 to utilize data provided by themultimodal sensor 104 related to a skeletal framework or profile of oneor more members of the audience, as generated by the depth data providedby the multimodal sensor 104. In some examples, the pose identifier 404utilizes coordinate information (e.g., the coordinate 302 of FIG. 3)provided by the example people analyzer 206 to focus an analysis on aparticular portion of the data provided by the multimodal sensor 104known to include (or at least previously include) a person. The examplepose identifier 304 uses the skeletal profile to determine or estimate apose (e.g., facing away, facing towards, looking sideways, lying down,sitting down, standing up, etc.) and/or posture (e.g., hunched over,sitting, upright, reclined, standing, etc.) of a detected audiencemember (e.g., the audience member 110 of FIG. 1). Poses that indicate afaced away position from the primary media device 102 (e.g., a bowedhead, looking away, etc.) generally indicate lower levels of engagementwith the primary media device 102. Upright postures (e.g., on the edgeof a seat) indicate more engagement with the primary media device 102.The example pose identifier 404 of FIG. 4 also detects changes in poseand/or posture, which may be indicative of more or less engagement withthe primary media device 102 (e.g., depending on a beginning and endingpose and/or posture).

Additionally or alternatively, the example pose identifier 204 of FIG. 4determines whether the audience member is making a gesture reflecting anemotional state, a gesture intended for a gaming control technique, agesture to control the primary media device 102, and/or identifies thegesture. Gestures indicating emotional reaction (e.g., raised hands,first pumping, etc.) indicate greater levels of engagement with theprimary media device 102. The example engagement level calculator 00 ofFIG. 4 determines that different poses, postures, and/or gesturesidentified by the example pose identifier 404 of FIG. 4 are more or lessindicative of engagement with, for example, a current media presentationvia the primary media device 102 by, for example, comparing theidentified pose, posture, and/or gesture to a look up table havingengagement scores assigned to the corresponding pose, posture, and/orgesture. Using this information, the example pose identifier 404 of FIG.4 calculates a likelihood that the corresponding audience member isengaged with the primary media device 102 for each frame (e.g., or someset of frames) of the media. Similar to the example eye tracker 402 ofFIG. 4, the example pose identifier 404 of FIG. 4 can combine theindividual likelihoods of engagement for multiple frames and/or audiencemembers to generate a collective likelihood for one or more periods oftime and/or can calculate a percentage of time in which poses, postures,and/or gestures indicate the audience member(s) (collectively and/orindividually) are engaged with the primary media device 102.

In the illustrated example of FIG. 4, the engagement level calculator400 includes an audio detector 406 to utilize audio information providedby the multimodal sensor 104. The example audio detector 406 of FIG. 4uses, for example, directional audio information provided by amicrophone array of the multimodal sensor 104 to determine a likelihoodthat the audience member is engaged with the primary media device 102.For example, a person that is speaking loudly or yelling (e.g., towardthe primary media device 102) may be interpreted by the audio detector406 of FIG. 4 as more likely to be engaged with the primary media device102 than someone speaking at a lower volume (e.g., because that personis likely having a conversation).

Further, speaking in a direction of the primary media device 102 (e.g.,as detected by the directional microphone array of the multimodal sensor104) may be indicative of a higher level of engagement with the primarymedia device 102. Further, when speech is detected but only one audiencemember is present, the example audio detector 406 of FIG. 4 may creditthe audience member with a higher level engagement. Further, when themultimodal sensor 104 is located proximate to the primary media device102, if the multimodal sensor 104 detects a higher (e.g., above athreshold) volume from a person, the example audio detector 406 of FIG.4 determines that the person is more likely facing the primary mediadevice 102. This determination may be additionally or alternatively madeby combining data from the camera of a video sensor.

In some examples, the spoken words from the audience are detected andcompared to the context and/or content of the media (e.g., to the audiotrack) to detect correlation (e.g., word repeats, actors names, showtitles, etc.) indicating engagement with the primary media device 102. Aword related to the context and/or content of the media is referred toherein as an ‘engaged’ word.

The example audio detector 406 of FIG. 4 uses the audio information tocalculate an engagement likelihood for frames of the media. Similar tothe example eye tracker 402 of FIG. 4 and/or the example pose identifier404 of FIG. 4, the example audio detector 406 of FIG. 4 can combineindividual ones of the calculated likelihoods to form a collectivelikelihood for one or more periods of time and/or can calculate apercentage of time in which voice or audio signals indicate the audiencemember(s) are paying attention to the primary media device 102.

In the illustrated example of FIG. 4, the engagement level calculator400 includes a position detector 408, which uses data provided by themultimodal sensor 104 (e.g., the depth data) to determine a position ofa detected audience member relative to the multimodal sensor 104 and,thus, the primary media device 102. In some examples, the positiondetector 408 utilizes coordinate information (e.g., the coordinate 302of FIG. 3) provided by the example people analyzer 206 to focus ananalysis on a particular portion of the data provided by the multimodalsensor 104 known to include (or at least previously include) a person.The example position detector 408 of FIG. 4 uses depth information(e.g., provided by the dot pattern information generated by the laser ofthe multimodal sensor 104) to calculate an approximate distance (e.g.,away from the multimodal sensor 104 and, thus, the primary media device102 located adjacent or integral with the multimodal sensor 104) atwhich an audience member is detected. The example position detector 408of FIG. 4 treats closer audience members as more likely to be engagedwith the primary media device 102 than audience members located fartheraway from the primary media device 102.

Additionally, the example position detector 408 of FIG. 4 uses dataprovided by the multimodal sensor 104 to determine a viewing angleassociated with each audience member for one or more frames. The exampleposition detector 408 of FIG. 4 interprets a person directly in front ofthe primary media device 102 as more likely to be engaged with theprimary media device 102 than a person located to a side of the primarymedia device 102. The example position detector 408 of FIG. 4 uses theposition information (e.g., depth and/or viewing angle) to calculate alikelihood that the corresponding audience member is engaged with theprimary media device 102. The example position detector 408 of FIG. 4takes note of a seating change or position change of an audience memberfrom a side position to a front position as indicating an increase inengagement. Conversely, the example position detector 408 of FIG. 4takes note of a seating change or position change of an audience memberfrom a front position to a side position as indicating a decrease inengagement. Similar to the example eye tracker 402 of FIG. 4, theexample pose identifier 404 of FIG. 4, and/or the example audio detector406 of FIG. 4, the example position detector 408 of FIG. 4 can combinethe calculated likelihoods of different (e.g., consecutive) frames toform a collective likelihood that the audience member is engaged withthe primary media device 102 and/or can calculate a percentage of timein which position data indicates the audience member(s) are payingattention to the primary media device 102.

The example engagement level calculator 400 of FIG. 4 includes asecondary device detector 410. The example secondary device detector 410of FIG. 4 uses detections of light patterns and/or glows in image data(e.g., data provided by the multimodal sensor 104) to (1) determinewhether the audience member 110 is interacting with a secondary mediadevice (e.g., the first secondary media device 112 of FIG. 1), (2) toidentify a type of the secondary media device being used by the audiencemember, (3) to identify the secondary media device itself, (4) to detectpresence of a secondary media device (e.g., the first secondary mediadevice 112 and/or the second secondary media device 114 of FIG. 1),and/or (5) to determine an engagement level based on the detectedinteraction with the secondary media device and/or an effect on anengagement level for the audience member 110 already calculated by, forexample, one or more of the other components 402-408 of the engagementlevel calculator 400. The light patterns and/or glows detected in theimage data are referred to herein as light information. In someexamples, the secondary device detector 410 utilizes coordinateinformation (e.g., the coordinate 302 of FIG. 3) provided by the examplepeople analyzer 206 to focus a search for the light information on aparticular portion of the data provided by the multimodal sensor 104known to include (or at least previously include) a person. The examplesecondary device detector 410 of FIG. 4 is described in detail below inconnection with FIGS. 5-7.

In some examples, the engagement level calculator 400 bases individualones of the engagement likelihoods and/or scores on particularcombinations of detections from different ones of the eye tracker 402,the pose identifier 404, the audio detector 406, the position detector408, the secondary device detector 410, and/or other component(s). Forexample, the engagement level calculator 400 of FIG. 4 generates arelatively high engagement likelihood and/or score for a combination ofthe eye tracker 402 determining that the audience member 110 is lookingat the primary media device 102 and the secondary device detector 410determining that the audience member 110 is not interacting with thefirst secondary media device 112. Additionally or alternatively, theexample engagement level calculator 400 of FIG. 4 generates a relativelylow engagement likelihood and/or score for a combination of the eyetracker 402 determining that the audience member 110 is looking awayfrom the primary media device 102 and the secondary device detector 410determining that the audience member 110 is interacting with the firstsecondary media device 112.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates a relatively high engagement likelihood and/orscore for a combination of the pose identifier 404 determining that theaudience member 110 is making a gesture known to be associated with thevideo game system 108 and the secondary device detector 410 determiningthat the audience member 110 is not interacting with the first secondarymedia device 112. Additionally or alternatively, the example engagementlevel calculator 400 of FIG. 4 generates a relatively low engagementlikelihood and/or score for a combination of the pose identifier 404determining that the audience member 110 is sitting in a hunched overpose and the secondary device detector 410 determining that the audiencemember 110 is interacting with the first secondary media device 112and/or the second secondary media device 114.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates a relatively high engagement likelihood and/orscore for a combination of the audio detector 406 determining that theaudience member 110 is quiet and the secondary device detector 410determining that the audience member 110 is not interacting with thefirst secondary media device 112. Additionally or alternatively, theexample engagement level calculator 400 of FIG. 4 generates a relativelylow engagement likelihood and/or score for a combination of the audiodetector determining that the audience member 110 is speaking softly andthe secondary device detector 410 determining that the audience member110 is interacting with the first secondary media device 112.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates a relatively high engagement likelihood and/orscore for a combination of the position detector 408 determining thatthe audience member 110 is located directly in front of the primarymedia device 102 and four (4) feet away from the primary media device102 and the secondary device detector 410 determining that the audiencemember 110 is not interacting with the first secondary media device 112.Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates a relatively low engagement likelihood and/orscore for a combination of the position detector 408 determining thatthe audience member 110 is located at an obtuse angle from the primarymedia device 102 and the secondary device detector 410 determining thatthe audience member 110 is interacting with the first secondary mediadevice 112.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates a relatively high engagement likelihood and/orscore for a combination of the position detector 408 determining thatthe audience member 110 is located directly in front of the primarydevice 102 and the secondary device detector 410 determining that theaudience member 110 is more than a threshold distance (e.g., three (3)feet) from the second secondary media device 114. Additionally oralternatively, the example engagement level calculator 400 of FIG. 4generates a relatively low engagement likelihood and/or score for acombination of the position detector 408 determining that the audiencemember 110 is located at an obtuse angle from the primary media device102 and the secondary device detector 410 determining that the audiencemember is less than a threshold distance away from the second secondarymedia device 114.

Further, in some examples, the engagement level calculator 400 combinesor aggregates the individual likelihoods and/or engagement scoresgenerated by the eye tracker 402, the pose identifier 404, the audiodetector 406, the position detector 408, and/or the secondary devicedetector 410 to form an aggregated likelihood for a frame or a group offrames of media (e.g. as identified by the media detector 202 of FIG. 2)presented by the primary media device 102. The aggregated likelihoodand/or percentage is used by the example engagement level calculator 400of FIG. 4 to assign an engagement level to the corresponding framesand/or group of frames. In some examples, the engagement levelcalculator 400 averages the generated likelihoods and/or scores togenerate the aggregate engagement score(s). Alternatively, the exampleengagement level calculator 400 of FIG. 4 calculates a weighted averageof the generated likelihoods and/or scores to generate the aggregateengagement score(s). In such instances, configurable weights areassigned to different ones of the detections associated with the eyetracker 402, the pose identifier 404, the audio detector 406, theposition detector 408, and/or the secondary device detector 410.

Moreover, the example engagement level calculator 400 of FIG. 4 factorsan attention level of some identified individuals (e.g., members of theexample household of FIG. 1) more heavily into a calculation of acollective engagement level for the audience more than othersindividuals. For example, an adult family member such as a father and/ora mother may be more heavily factored into the engagement levelcalculation than an underage family member. As described above, theexample meter 106 of FIGS. 1 and/or 2 is capable of identifying a personin the audience as, for example, a father of a household. In someexamples, an attention level of the father contributes a firstpercentage to the engagement level calculation and an attention level ofthe mother contributes a second percentage to the engagement levelcalculation when both the father and the mother are detected in theaudience. For example, the engagement level calculator 400 of FIG. 4uses a weighted sum to enable the engagement of some audience members tocontribute to a “whole-room” engagement score than others. The weightedsum used by the example engagement level calculator 400 of FIG. 4 can begenerated by Equation 1 below.

$\begin{matrix}{{RoomScore} = \frac{\begin{matrix}{{{DadScore}*(0.3)} + {{MomScore}*(0.3)} +} \\{{{TeenagerScore}*(0.2)} + {{ChildScore}*(0.1)}}\end{matrix}}{\begin{matrix}{{FatherScore} + {MotherScore} +} \\{{TeenagerScore} + {ChildScore}}\end{matrix}}} & {{Equation}\mspace{14mu} 1}\end{matrix}$

The above equation assumes that all members of a family are detected.When only a subset of the family is detected, different weights may beassigned to the different family members. Further, when an unknownperson is detected in the room, the example engagement level calculator400 of FIG. 4 assigns a default weight to the engagement scorecalculated for the unknown person. Additional or alternativecombinations, equations, and/or calculations are possible.

Engagement levels generated by the example engagement level calculator400 of FIG. 4 are stored in an engagement level database 412. Content ofthe example engagement level database 412 of FIG. 4 are periodicallyand/or aperiodically exported to, for example, the data collectionfacility 216.

While an example manner of implementing the behavior monitor 208 of FIG.2 has been illustrated in FIG. 4, one or more of the elements, processesand/or devices illustrated in FIG. 4 may be combined, divided,re-arranged, omitted, eliminated and/or implemented in any other way.Further, the example engagement level calculator 400, the example eyetracker 402, the example pose identifier 404, the example audio detector406, the example position detector 408, the example secondary devicedetector 410, and/or, more generally, the example behavior monitor 208of FIG. 4 may be implemented by hardware, software, firmware and/or anycombination of hardware, software and/or firmware. Thus, for example,any of the example engagement level calculator 400, the example eyetracker 402, the example pose identifier 404, the example audio detector406, the example position detector 408, the example secondary devicedetector 410, and/or, more generally, the example behavior monitor 208of FIG. 4 could be implemented by one or more circuit(s), programmableprocessor(s), application specific integrated circuit(s) (ASIC(s)),programmable logic device(s) (PLD(s)) and/or field programmable logicdevice(s) (FPLD(s)), field programmable gate array (FPGA), etc. When anyof the apparatus or system claims of this patent are read to cover apurely software and/or firmware implementation, at least one of theexample engagement level calculator 400, the example eye tracker 402,the example pose identifier 404, the example audio detector 406, theexample position detector 408, the example secondary device detector410, and/or, more generally, the example behavior monitor 208 of FIG. 4are hereby expressly defined to include a tangible computer readablestorage medium such as a storage device (e.g., memory) or an opticalstorage disc (e.g., a DVD, a CD, a Bluray disc) storing the softwareand/or firmware. Further still, the example behavior monitor 208 of FIG.4 may include one or more elements, processes and/or devices in additionto, or instead of, those illustrated in FIG. 4, and/or may include morethan one of any or all of the illustrated elements, processes anddevices.

FIG. 5 is a block diagram of an example implementation of the examplesecondary device detector 410 of FIG. 4. The example secondary devicedetector 410 of FIG. 5 analyzes image data of the environment 100 todetermine whether audience member(s) are interacting with a secondarymedia device (e.g., a media presentation device other than the primarymedia device 102 of FIG. 1). In particular, the example secondary devicedetector 410 of FIG. 5 searches for light patterns projected onto anobject (e.g., a face) and/or glows emanating from a device. For example,the first secondary media device 112 of FIG. 1 with which the audiencemember 110 is interacting projects light onto the face of the audiencemember 110 and, in doing so, creates a particular light pattern on theface of the audience member 110. The example secondary device detector410 of FIG. 5 detects the light pattern and uses data related to thedetected light pattern to, for example, calculate an engagement level ofthe audience member 110 for the primary media device 102 in view of theinteraction with the secondary media device 112, identify a type of thesecondary media device 112, and/or identify the secondary media device112 itself (e.g., by brand and/or model). Additionally or alternatively,the second secondary media device 114 in the environment 100 is poweredon and producing light that forms a glow emanating from a screen of thedevice 112. A similar glow is also generated by the first secondarymedia device 112. The example secondary device detector 410 of FIG. 5detects the glow(s) and uses data related to the glow(s) to, forexample, determine that the first and/or second secondary media devices112, 114 are present (and powered on) in the environment.

To detect light signatures corresponding to light patterns projected onan object, the example secondary device detector 410 of FIG. 5 includesa signature generator 500 to generate, receive, obtain, and/or updatelight signatures representative of light patterns known to correspondto, for example, a type of media presentation device and/or a particularmedia presentation device. In some examples, the light signaturesgenerated, received, obtained, and/or updated by the signature generator500 each correspond to a pattern of light on a body of a person (e.g., aface of a person) projected by the corresponding type of mediapresentation device and/or particular media presentation device. In someexamples, the light signatures include multi-dimensional characteristicsor measurements representative of, for example, brightness, hue, and/orcontrast values of proximate portions of image data. For example, alight signature may include a series of expected brightness valuesrelative to each other (e.g., brightness deltas) that follow a contourof a human face and/or head. In such instances, different ones of thelight signatures correspond to different orientations of the human faceand/or head relative to, for example, the multimodal sensor 104providing the corresponding image data. In some examples, the lightsignatures are representative of expected differences between a firstamount of light projected on a face looking at nearby screen (e.g., atablet being held in front of the face) and a second amount of lightfound on a different body part, such as a shoulder or chest of theperson. In some examples, the light signatures include contrast valuesindicative of difference(s) between the brightness found on a face andthe brightness of an area surrounding the face, such as an ambientamount of brightness. In some examples, the light signatures includefrequency spectrums indicative of different hues found at, for example,different depths and/or other image locations. In some examples, thelight signatures include intensity graphs representative of lightmeasurements in different portions of the image data. The example lightsignatures can include and/or be based on additional or alternativetypes of data, measurement(s), and/or characteristic(s).

In some examples, the example signature generator 500 of FIG. 5 utilizestest results associated with light patterns measured during, forexample, a laboratory analysis performed to determine characteristics oflight patterns produced by displays of different media presentationdevices. For example, a first test may determine a first light signaturefor a first type of media presentation device being used (e.g., heldand/or looked at) by a person. The first light signature corresponds to,for example, light projected from a display of a certain type of mediapresentation device onto a face and/or other body part of a person. Inanother example, a second test may determine a second light signaturefor a particular brand and/or model of media presentation being used bya person. The second light signature corresponds to, for example, lightprojected from a display of a particular brand and/or model of mediapresentation device onto a face and/or other body part of a person. Inaddition to or in lieu of utilizing the test results to generate thelight signatures, the example signature generator 500 of FIG. 5 mayimplement one or more algorithms to determine an expected light patterngenerated by, for example, a media presentation device being held by aperson. The example signature generator 500 can utilize any additionalor alternative techniques or components to generate light signaturescorresponding to, for example, the audience member 110 of FIG. 2interacting with a secondary media device.

In the illustrated example of FIG. 5, the light signature generator 500generates different light signatures for different lighting conditionsof an environment in which the example secondary device detector 410 ofFIG. 5 is implemented. The light patterns to be detected by the examplesecondary device detector 410 of FIG. 5 may be more difficult or lessdifficult to detect in some lighting conditions than others. In theillustrated example of FIG. 5, the light signature generator 500generates sets of light signatures, each corresponding to one of aplurality of different lighting conditions. A set generated by theexample light signature generator 500 of FIG. 5 includes, for example,light signature(s) corresponding to device type detection (e.g.,identification of type(s) of device(s)) and/or light signature(s)corresponding to device detection (e.g., identification of particulardevice(s)).

The example light signature generator 500 of FIG. 5 generates a firstset of light signatures 504 for use when the environment 100 is full ofdaylight. To generate such a first set of signatures 504, a testenvironment is filled with daylight and light pattern detection testsare executed on a test subject located in the naturally lit testenvironment. Further, the example light signature generator 500 of FIG.5 generates a second set of light signatures 506 for use when theenvironment 100 is full of artificial light (e.g., from one or morelamps and/or the glow from the primary media device 102). To generatesuch a second set of signatures 506, a test environment is filled withartificial light and light pattern detections tests are executed on atest subject located in the artificially lit test environment. Further,the example light signature generator 500 of FIG. 5 generates a thirdset of light signatures 508 for use when the environment 100 includes amixture of daylight and artificial light. To generate such a third setof signatures 508, a test environment is filled with a combination ofdaylight and artificial light and light pattern detections tests areexecuted on a test subject located in the naturally and artificially littest environment. Further, the example light signature generator 500 ofFIG. 5 generates a fourth set of light signatures 510 for use when theenvironment 100 includes a first ratio of daylight to artificial light.To generate such a fourth set of signatures 510, a test environment isfilled with a first amount of daylight and a second amount of artificiallight and light pattern detection tests are executed on a test subjectlocated in the naturally and artificially lit test environment. Further,the example light signature generator 500 of FIG. 5 generates a fifthset of light signatures 512 for use when the environment 100 includes asecond ratio of daylight to artificial light different than the firstratio associated with the fourth set of light signatures 510. Togenerate such a fifth set of signatures 512, a test environment isfilled with a third amount of daylight and a fourth amount of artificiallight and light pattern detection tests are executed on a test subjectlocated in the naturally and artificially lit test environment. Further,the example light signature generator 500 of FIG. 5 generates a sixthset of light signatures 514 for use when the environment 100 includes afirst total amount of light. To generate such a sixth set of signatures514, a test environment is filled with the first amount of total light(e.g., all daylight, all artificial light, or a combination of daylightand artificial light) and light pattern detection tests are executed ona test subject located in the test environment. Further, the examplelight signature generator 500 of FIG. 5 generates a seventh set of lightsignatures 516 for use when the environment 100 includes a second totalamount of light. To generate such a seventh set of signatures 516, atest environment is filled with the second amount of total light (e.g.,all daylight, all artificial light, or a combination of daylight andartificial light) and light pattern detection tests are executed on atest subject located in the test environment. Further, the example lightsignature generator 500 of FIG. 5 generates an eighth set of lightsignatures 518 for use at a first time of day. To generate such a eighthset of signatures 518, light pattern detection tests are executed on atest subject located in the test environment at the first time of day.Further, the example light signature generator 500 of FIG. 5 generates aninth set of light signatures 520 for use at a second time of day. Togenerate such a ninth set of signatures 520, light pattern detectiontests are executed on a test subject located in the test environment atthe second time of day. The example light signature generator 500 ofFIG. 5 can generate additional or alternative set(s) 510 of lightsignatures corresponding to, for example, different lighting conditions,different types of artificial light sources (e.g., CFL, incandescent,etc.) and/or other characteristics of the environment 100. Any or all ofthe above tests can be performed for different devices to develop setsof signatures for different devices types, manufacturers, models, etc.

FIG. 6 illustrates an example implementation of the first set of lightsignatures 504 of FIG. 5. The example first set of light signatures 504is shown in a table 600 having a signature column 602 and an identifiercolumn 604. In the example of FIG. 6, entries in the identifier column604 correspond to a respective entry in the signature column 602. Thatis, each one of the signatures in the signature column 602 correspondsto an identifier in the identifier column 604 that provides identifyinginformation indicative of, for example, a usage detection, the type ofdevice, and/or the particular device (e.g., by model, by, manufacturer,etc.) known to project a light pattern similar to the correspondinglight signature of the signature column 602. In the example of FIG. 6, afirst portion 606 of the table 600 includes identifiers of types ofmedia devices. Thus, signatures of the first portion 606 are known tocorrespond to a particular respective type of media device such as, forexample, a tablet, a mobile telephone, a laptop computer, a desktopcomputer, etc. A second portion 608 of the table 600 of FIG. 6 includesan identifier of a particular brand of media device, such as Apple®products. A third portion 610 of the table 600 of FIG. 6 includesidentifiers of particular media devices as identified by, for example,product name and/or model. In some examples, a type of mediapresentation device can be inferred from the brand name and/or modelidentification. For example, if the example table 600 of FIG. 6 is usedto identify the first secondary device 112 as an Apple iPad®, it can beinferred that the first secondary device 112 is a tablet. Suchassociations between specific products and device types can be storedin, for example, the example table 600 of FIG. 6. Additional oralternative types of information can be included in the example table600 of FIG. 6. In the illustrated example, tables similar to the exampletable 600 of FIG. 6 are used to implement the example sets of lightsignatures 504-520 of the example light signature database 502 of FIG.5.

The example secondary device detector 410 of FIG. 5 includes a lightcondition detector 522 that receives signal(s) from the example ambientlight sensor 204 of FIG. 2. The example ambient light sensor 204 of FIG.2 provides the light condition detector 522 with data related to thelight present in the environment 100. The example light conditiondetector 522 processes the signal(s) provided by the ambient lightsensor 204 to calculate an amount of natural light in the environment100, an amount of artificial light in the environment 100, a ratio ofnatural light to artificial light in the environment 100, a total amountof light in the environment 100, and/or any other suitable lightingcharacteristics or conditions.

The example secondary device detector 410 of FIG. 5 includes a signatureset selector 524 to select one or more of the sets of light signatures504-520 of the example light signature database 502. In the illustratedexample of FIG. 5, the signature set selector 524 uses indication(s)generated by the example light condition detector 522 of FIG. 5 to makea selection from the light signature database 502. For example, when thelight condition detector 522 indicates that the environment 100 is fullof natural light, the example signature set selector 524 selects thefirst set of light signatures 504. Additionally or alternatively, theexample signature set selector 524 of FIG. 5 uses a time of day to makea selection from the light signatures database 502. For example, at thefirst time of day mentioned above in connection with the light signaturedatabase 502, the example signature set selector 524 selects the eighthset of light signatures 518. In some examples, the signature setselector 524 selects more than one set of light signature depending on,for example, how many of the lighting conditions associated with thelight signature database 502 are met by current conditions of theenvironment 100.

The example secondary device detector 410 of FIG. 5 includes a lightpattern identifier 526 to detect light patterns projected onto object(s)in the environment 100, such as the face of the audience member 110. Asdescribed above, the example device usage indicator 410 receives imagedata (e.g., frames of three-dimensional data and/or two-dimensionaldata) representative of the environment 100. The example light patternidentifier 526 searches the received image data for instances in which alocalized light pattern is found on one or more body parts of a person.A localized light pattern is detected by, for example, identifyingportions of the image data including intensities (e.g., on a grayscale)that sharply contrast with immediate surroundings. In some examples, thelight pattern identifier 526 searches the entire frames of image datafor the localized light patterns. Additionally or alternatively, theexample light pattern identifier 526 of FIG. 5 utilizes face detectionsgenerated by, for example, the people analyzer 206 of FIG. 2. Forexample, when the people analyzer 206 provides a coordinate (e.g., thecoordinate 302 of FIG. 3) to the example device usage indicator 410 ofFIG. 5, the example light pattern identifier 526 may focus a search forlocalized light patterns to a portion of the image data corresponding tothe received coordinate. For example, the light pattern identifier 526of FIG. 5 searches a circular or rectangular (or any other suitableshape) area of image data centered on the received coordinatecorresponding to a face detection. In doing so, the example lightpattern identifier 526 saves computational resources by avoidingperformance of a search of the entire frame of image data. In someexamples, the light pattern identifier 526 is triggered in response toreceiving a face detection from the people analyzer 206. In someexamples, the light pattern identifier 526 is periodically triggered inresponse to a scheduled event (e.g., according to a timer).

The example light pattern identifier 526 of FIG. 5 provides image datacorresponding to detected light pattern(s) found on, for example, theaudience member 110 to a comparator 530. Further, the example signatureset selector 524 provides comparator 530 with the selected set(s) oflight signatures and/or an instruction of which set(s) of lightsignatures were selected. As described above, more than one set of lightsignatures are selected by the example signature set selector 524 whenthe current lighting condition of the environment satisfies more thanone of the sets of the light signatures 504-520. The example comparator530 of FIG. 5 compares the detected light signature(s) found on theaudience member 110 to the light signatures of the selected set(s) oflight signatures. In the illustrated example, the comparator 530generates a similarity score indicative of how closely the detectedlight patterns in the environment 100 match the selected lightsignatures. The similarly scores generated by the example comparator 530are of any suitable format such as, for example, a percentage or a scalevalue.

The example secondary device detector 410 of FIG. 5 includes a deviceidentifier 532 to receive the similarity scores from the comparator 530.The example device identifier 532 of FIG. 5 manages one or morethresholds to which the similarity scores are compared. In theillustrated example, the threshold(s) used by the device identifier 532represent how closely the detected light patterns need to match theselected light signature(s) to be considered as corresponding to therespective device type and/or particular product. In some examples, thedevice identifier 532 includes different thresholds for different onesof the light signatures. For example, the device identifier 532 of FIG.5 uses a first threshold for the first portion 606 of the example table600 of FIG. 6 and a second, different threshold for the third portion610 of the table 600. In some examples, the first threshold is greaterthan the second threshold, thereby requiring the detected light patternsto match the signatures of the third portion 610 more closely than thesignatures of the first portion 606 to be considered a detection of therespective device type and/or particular product. In some examples, thedevice identifier 532 includes a global threshold to be applied to eachof the similarity scores received from the comparator 530.

If the example device identifier 532 of FIG. 5 determines that asimilarity score generated by the example comparator 530 meets orexceeds one or more of the appropriate usage thresholds, the exampledevice identifier 532 generates a usage indicator that the correspondinglight signature(s) are present in the analyzed image data of theenvironment 100. As described above, such an indication is indicativeof, for example, interaction with a particular type of secondary mediadevice and/or a particular secondary media device. For example, thedevice identifier 532 of FIG. 5 generates an indicator that the audiencemember 110 of FIG. 1 is interacting with the first secondary mediadevice 112 and an identifier (e.g., device type, product name,manufacturer, model number, etc.) of the first secondary media device112 when the corresponding threshold has been met or exceeded. In someexamples, the device identifier 532 of FIG. 5 generates a confidencelevel in conjunction with the usage indications representative of adegree at which the corresponding threshold was exceeded by thesimilarity score. That is, when the similarity of the detected lightpattern to the selected light signature exceeds the correspondingthreshold by a first degree, the example device identifier 532 of FIG. 5assigns a first confidence level to the generated usage indication.Further, when the similarity of the detected light pattern to theselected light signature exceeds the corresponding threshold by a seconddegree lesser than the first degree, the example device identifier 532of FIG. 5 assigns a second confidence level less than the firstconfidence level to the generated usage indication.

The example secondary device detector 410 of FIG. 5 also includes a glowidentifier 528 to detect glow(s) present in the environment 100. Theexample glow identifier 528 of FIG. 5 identifies instances in the imagedata of localized brightness that correspond to, for example, an amountof light emanating from a display of the second secondary media device114 of FIG. 1. In some examples, the glow identifier 528 identifiesportions of the image data of a certain size that include higherbrightness values surrounded by lower brightness values. In someexample, the glow identifier 528 generates and/or utilizes an intensitygraph of the room representative of light characteristic values (e.g.,brightness, hue, contrast, etc.) to identify the portions likely tocorrespond to a glow emanating from a screen. In some examples, the glowidentifier 528 uses location information associated with, for example, aface detection to focus an analysis on a designated portion of the imagedata, such as a circle or rectangular surrounding the coordinate 302 ofFIG. 3. In some examples, the glow identifier 528 is periodicallytriggered in response to a scheduled event (e.g., according to a timer).

In the illustrated example, the glow identifier 528 generates a presenceindication when a glow from a secondary media device is detected. Thus,the presence indications generated by the example glow identifier 528indicate that a secondary media device is present in the environment100. For example, the glow identifier 528 of FIG. 5 determines that thesecond secondary media device 114 is present in the environment 100.Additionally or alternatively, the example glow identifier 528 of FIG. 5determines that the first secondary media device 112 is present in theenvironment 100 when the glow emanating from the first secondary mediadevice 112 is detectable (e.g., according to the orientation of thefirst secondary media device 112 relative to the multimodal sensor 104).In some examples, the glow identifier 528 of FIG. 5 generates aconfidence level associated with the determination that the secondsecondary media device 114 is present in the environment 100. Theconfidence level is based on, for example, a similarity between thecollected data and the image characteristics known to correspond to aglow emanating from a display.

In some examples, the glow identifier 528 of FIG. 5 measures a distancebetween a detected glow and detected audience member(s). For example,when the glow identifier 528 identifies a glow emanating from the secondsecondary media device 114 of FIG. 1, the glow identifier 528 determinesa distance between the second secondary media device 114 and theaudience member 110. In the illustrated example, the glow identifier 528utilizes information generated by the people analyzer 206 of FIG. 2,such as the example coordinate 302 of FIG. 3 indicative of a location ofthe audience member 110 in the environment 100. In some examples, theglow identifier 528 and/or the example engagement level calculator 400of FIG. 4 base an engagement level on the distance between the detectedglow and the audience member 110. For example, a first distance betweenthe detected glow and the audience member 110 is indicative of firstlevel of engagement and a second distance between the detected glow andthe audience member 110 is indicative of a second level of engagement.That is, in some examples, the second secondary media device 114 isconsidered more likely to draw attention away from the primary mediadevice 102 when the second secondary media device 114 is close to theaudience member 110.

In some examples, the glow identifier 528 detects changes in the glowemanating from, for example, the second secondary media device 114 ofFIG. 1 (e.g., over a period of time, such as three (3) seconds and/orthe corresponding amount of frames). The example glow identifier 528and/or the example engagement level calculator 400 of FIG. 4 interpretschanges in the glow as indications that the second secondary mediadevice 114 is currently presenting media and, for example, is morelikely to draw the attention of the audience member 110 (e.g., than astatic display).

In the illustrated example, data generated by the example deviceidentifier 532 (e.g., the usage indication(s) and/or the correspondingconfidence level(s)) and/or data generated by the example glowidentifier 528 (e.g., the presence indication(s), the correspondingconfidence level(s), and/or the distances between the present secondarymedia device(s) and the audience member(s)) are used to calculate anengagement level for the audience member 110 with respect to, forexample, the primary media device 102. In some examples, an engagementlevel for the audience member 110 calculated by other component(s) ofthe engagement level calculator 400 (e.g., the eye tracker 402, the poseidentifier 404, the audio detector 406 and/or the position detector 408)can be adjusted (e.g., decreased or increased) when the example usagedetector 532 of FIG. 5 determines that the audience member 110 isinteracting with the first secondary media device 112 and/or that thefirst and/or secondary media devices 112, 114 are present in theenvironment 100. In some examples, the amount of adjustment in thealready calculated engagement level depends on, for example, thecorresponding confidence level generated by the example deviceidentifier 532 and/or glow identifier 528.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 generates an engagement level for the audience member 110with respect to the primary media device 102 based solely on the usageindications generated by the example device identifier 532 and/or basedsolely on the presence indications generated by the example glowidentifier 528. For example, the engagement level calculator 400 of FIG.4 assigns a first engagement level to the audience member 110 for theprimary media device 102 when the example device identifier 532indicates that the audience member 110 is interacting with the firstsecondary media device 112 and a second engagement level when theexample device identifier 532 indicates that the audience member 110 isnot interacting with the first secondary media device 112. Additionallyor alternatively, the example engagement level calculator 400 of FIG. 4assigns a third engagement level to the audience member 110 with respectto the primary media device 102 when the example device identifier 534indicates that the first secondary media device 112 is a first type ofmedia device (e.g., a tablet) and a fourth engagement level when thefirst secondary media device 112 is a second type of media device (e.g.a mobile phone) different from the first type of media device.Additionally or alternatively, the example engagement level calculator400 of FIG. 4 assigns a fifth engagement level to the audience member110 with respect to the primary media device 102 when the example deviceidentifier 534 indicates that the first secondary media device 112 is afirst media device (e.g., an Apple® iPad®) and a sixth engagement levelwhen the secondary media device 112 is a second media device (e.g., anApple® iPhone®).

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 assigns a seventh engagement level to the audience member110 for the primary media device 102 when the example glow identifier532 indicates that the second secondary media device 114 is present inthe environment 100 and a eighth engagement level when the example glowidentifier 528 indicates that the second secondary media device 114 isnot present (or powered off) in the environment 100.

Additionally or alternatively, the example engagement level calculator400 of FIG. 4 assigns a ninth engagement level to the audience member110 based on the confidence level(s) associated with the generated usageindications. That is, the engagement level generated for the exampleaudience member 110 can depend on, for example, how closely the detectedlight pattern on the audience member 110 matches the corresponding lightsignature. Additionally or alternatively, the example engagement levelcalculator 400 of FIG. 4 assigns a tenth engagement level to theaudience member 110 based on the confidence level(s) associated with thegenerated presence indications. That is, the engagement level generatedfor the example audience member 110 can depend on, for example, howclosely the detected glow matches the light characteristics associatedwith a glow emanating from a secondary media device.

In some examples, an calculated engagement level already calculated forthe audience member 110 by other component(s) of the engagement levelcalculator 400 (e.g., the eye tracker 402, the pose identifier 404, theaudio detector 406 and/or the position detector 408) can be adjusted(e.g., increased or decreased) when the example device identifier 532 ofFIG. 5 determines that the audience member 110 is interacting with aparticular type of secondary media device and/or a particular secondarymedia device. In other words, the example audience member 110 may beconsidered less likely to be paying attention to the primary mediadevice 102 while interacting with a tablet than while interacting with alaptop computer. The amount of adjustment to the calculated engagementlevel depends on, for example, the corresponding confidence levelgenerated by the example device identifier 532.

In some examples, the engagement level calculator 400 combines differentindications (e.g., a first indication of a type of device and a secondindication of a particular device) to generate an aggregate engagementlevel for the audience member 110. In some examples, the secondarydevice detector 410 of FIG. 5 combines usage indication(s) generated bythe device identifier 532 with presence indication(s) generated by theglow identifier 528 to calculate an aggregate engagement level for theaudience member 110 with respect to the primary media device 102.Additionally or alternatively, the engagement level(s) calculated by theexample device identifier 532 and/or the glow identifier 528 can becombined with the engagement level(s) generated by, for example, the eyetracker 302, the pose identifier 304, the audio detector 406, theposition detector 408 and/or any other component.

While an example manner of implementing the secondary device detector410 of FIG. 4 has been illustrated in FIG. 5, one or more of theelements, processes and/or devices illustrated in FIG. 5 may becombined, divided, re-arranged, omitted, eliminated and/or implementedin any other way. Further, the example signature generator 500, theexample light condition detector 522, the example signature set selector524, the example light pattern identifier 526, the example glowidentifier 528, the example comparator 530, the example deviceidentifier 532 and/or, more generally, the example secondary devicedetector 410 of FIG. 5 may be implemented by hardware, software,firmware and/or any combination of hardware, software and/or firmware.Thus, for example, any of the example signature generator 500, theexample light condition detector 522, the example signature set selector524, the example light pattern identifier 526, the example glowidentifier 528, the example comparator 530, the example deviceidentifier 532 and/or, more generally, the example secondary devicedetector 410 of FIG. 5 could be implemented by one or more circuit(s),programmable processor(s), application specific integrated circuit(s)(ASIC(s)), programmable logic device(s) (PLD(s)) and/or fieldprogrammable logic device(s) (FPLD(s)), field programmable gate array(FPGA), etc. When any of the apparatus or system claims of this patentare read to cover a purely software and/or firmware implementation, atleast one of the example signature generator 500, the example lightcondition detector 522, the example signature set selector 524, theexample light pattern identifier 526, the example glow identifier 528,the example comparator 530, the example device identifier 532 and/or,more generally, the example secondary device detector 410 of FIG. 5 arehereby expressly defined to include a tangible computer readable storagemedium such as a storage device (e.g., memory) or an optical storagedisc (e.g., a DVD, a CD, a Bluray disc) storing the software and/orfirmware. Further still, the example secondary device detector 410 ofFIG. 5 may include one or more elements, processes and/or devices inaddition to, or instead of, those illustrated in FIG. 5, and/or mayinclude more than one of any or all of the illustrated elements,processes and devices.

While the example secondary device detector 410 of FIGS. 4 and/or 5 isdescribed above as implemented in the example meter 106 of FIGS. 1and/or 2, the example secondary device detector 410 or at least onecomponent of the example secondary device detector 410 can beimplemented in, for example, the first and/or second secondary mediadevices 112, 114 of FIG. 1. In such instances, components implemented inthe secondary media device(s) 112, 114 are in communication with themeter 106 and exchange data (e.g., in real time). In some examples, thelight condition detector 522 is implemented via a light sensor on thefirst secondary media device 112. In some examples, the light patternidentifier 526 is implemented via the first secondary media device 112and detects lights patterns on a face proximate the display of the firstsecondary media device 112. In some examples, the glow identifier 528 isimplemented via the second secondary media device 114 and detects theglow emanating from the display of the second secondary media device 114(when the display is on). Additional or alternative combinations ofcomponents of the example secondary device detector 410 are possible.

FIG. 7 is a flowchart representative of example machine readableinstructions for implementing the example secondary device detector 410of FIGS. 4 and/or 5. In the example of FIG. 7, the machine readableinstructions comprise a program for execution by a processor such as theprocessor 812 shown in the example processing platform 800 discussedbelow in connection with FIG. 8. The program may be embodied in softwarestored on a tangible computer readable storage medium such as a CD-ROM,a floppy disk, a hard drive, a digital versatile disk (DVD), a Blu-raydisk, or a memory associated with the processor 812, but the entireprogram and/or parts thereof could alternatively be executed by a deviceother than the processor 812 and/or embodied in firmware or dedicatedhardware. Further, although the example programs are described withreference to the flowchart illustrated in FIG. 7, many other methods ofimplementing the example secondary device detector 410 may alternativelybe used. For example, the order of execution of the blocks may bechanged, and/or some of the blocks described may be changed, eliminated,or combined.

As mentioned above, the example processes of FIG. 7 may be implementedusing coded instructions (e.g., computer readable instructions) storedon a tangible computer readable storage medium such as a hard diskdrive, a flash memory, a read-only memory (ROM), a compact disk (CD), adigital versatile disk (DVD), a cache, a random-access memory (RAM)and/or any other storage medium in which information is stored for anyduration (e.g., for extended time periods, permanently, brief instances,for temporarily buffering, and/or for caching of the information). Asused herein, the term tangible computer readable storage medium isexpressly defined to include any type of computer readable storagedevice and/or storage disc and to exclude propagating signals.Additionally or alternatively, the example processes of FIG. 7 may beimplemented using coded instructions (e.g., computer readableinstructions) stored on a non-transitory computer readable storagemedium such as a hard disk drive, a flash memory, a read-only memory, acompact disk, a digital versatile disk, a cache, a random-access memoryand/or any other storage medium in which information is stored for anyduration (e.g., for extended time periods, permanently, brief instances,for temporarily buffering, and/or for caching of the information). Asused herein, the term non-transitory computer readable storage medium isexpressly defined to include any type of computer readable storagedevice or storage disc and to exclude propagating signals. As usedherein, when the phrase “at least” is used as the transition term in apreamble of a claim, it is open-ended in the same manner as the term“comprising” is open ended. Thus, a claim using “at least” as thetransition term in its preamble may include elements in addition tothose expressly recited in the claim.

The example flowchart of FIG. 7 begins with an initiation of the deviceusage detector 700 which coincides with, for example, the example meter106 of FIG. 1 being powered on and/or otherwise activated (e.g., inresponse to the primary media device 102 being powered one) (block 700).In some instances, the initiation of the secondary device detector 410causes the example signature generator 500 of FIG. 5 to generate,update, and/or receive one or more light signatures. If the examplesignature generator 500 of FIG. 5 is triggered (block 702), the examplesignature generator 500 generates, updates, and/or receives lightsignatures and conveys the same to the example light signature database502 of FIG. 5 (block 704). The generated light signatures are organizedinto sets of light signatures 504-520 in the example database 502according to, for example, a lighting condition for which the respectivelight signatures are to be used. As described above, the lightsignatures of the database 502 correspond to light patterns known tocorrespond to a projection of light onto a person by a mediapresentation device.

The example light pattern identifier 526 of FIG. 5 obtains image datarepresentative of the environment 100 from, for example, the multimodalsensor of FIGS. 1 and/or 2 (block 706). The obtained image data includesthree-dimensional and/or two-dimensional data captured of theenvironment 100. The example light pattern identifier 526 analyzes theimage data to determine whether the environment 100 includes a localizedprojection of light onto an object, such as a body part of person (block708). In some examples, the analysis performed by the light patternidentifier 526 is focused on a portion of the image data correspondingto a face detected by the example people analyzer 206 as indicated inthe coordinate 302 of FIG. 3.

If the example light pattern identifier 526 determines that theenvironment 100 includes a localized light pattern projected on anobject (block 708), the example light condition sensor 522 of FIG. 5detects a lighting condition for the environment 100 corresponding tothe analyzed image data (block 710). For example, the light conditionsensor 522 determines that the environment 100 includes a certain totalamount of light, that the environment 100 includes a certain ratio ofnatural light to artificial light, and/or any other suitable lightingcondition characteristic(s). The example signature set selector 524 ofFIG. 5 uses the detected lighting condition of the environment 100 toselect one or more of the sets of light signatures 504-520 of thedatabase 502 (block 712). Thus, the example signature set selector 524selects the appropriate light signatures for comparison to the detectedlocalized light pattern identified by the example light patternidentifier 526.

The example comparator 530 of FIG. 5 compares the light signatures ofthe selected set(s) of light signatures to the detected light patterndetected on an object of the environment 100, such as the audiencemember 110 (block 714). The example comparator 530 generates a similarscore for each comparison representative of a degree (e.g., apercentage) of similarity between the respective ones of the lightsignatures and the detected localized light pattern. The similarityscores and information indicative of the corresponding light signaturesare provided to the example device identifier 532 of FIG. 5. The exampledevice identifier 532 compares the received similarity scores to one ormore thresholds to determine whether the detected light pattern in theimage data is similar enough to the respective light signatures toindicate an interaction with a secondary media device (block 716). Thecomparison(s) of the device identifier 532 generate usage indication(s)and/or corresponding confidence level(s) when the threshold(s) are metor exceeded. As described above, the data generated by the exampledevice identifier 532 is used by, for example, the engagement levelcalculator 400 of FIG. 4 to calculate a engagement level for theaudience member 110 with respect to the primary media device 102 of FIG.1.

In the example of FIG. 7, the example glow identifier 528 determineswhether a glow similar to light emanating from a secondary media deviceis present in the image data provided by, for example, the multimodalsensor 104 of FIG. 1 (block 718). If such a glow is detected (block718), the example glow identifier 528 measures a distance between thedetected glow and any detected audience members (e.g., according to thepeople analyzer 206 of FIG. 2) (block 720). Further, the example glowidentifier 528 detects one or more changes of the glow over a period oftime and/or a number of frames of image data (block 722). Further, theexample glow identifier 528 generates a presence indication toindicative, for example, that the second secondary media device 114 ofFIG. 1 is present in the environment 100 (block 724). As describedabove, the data generated by the example glow identifier 528 can be usedby, for example, the engagement level calculator 400 of FIG. 4 tocalculate a engagement level for the audience member 110 with respect tothe primary media device 102 of FIG. 1. Control then returns to block702.

FIG. 10 is a block diagram of an example processor platform 800 capableof executing the instructions of FIG. 7 to implement the examplesecondary device detector 410 of FIGS. 4 and/or 5 and/or executing theinstructions of FIG. 9 to implement the example media presentationdevice of FIG. 8. The processor platform 800 can be, for example, aserver, a personal computer, a mobile phone, a personal digitalassistant (PDA), an Internet appliance, a DVD player, a CD player, adigital video recorder, a BluRay player, a gaming console, a personalvideo recorder, a set-top box, an audience measurement device, or anyother type of computing device.

The processor platform 800 of the instant example includes a processor812. For example, the processor 812 can be implemented by one or morehardware processors, logic circuitry, cores, microprocessors orcontrollers from any desired family or manufacturer.

The processor 812 includes a local memory 813 (e.g., a cache) and is incommunication with a main memory including a volatile memory 814 and anon-volatile memory 816 via a bus 818. The volatile memory 814 may beimplemented by Synchronous Dynamic Random Access Memory (SDRAM), DynamicRandom Access Memory (DRAM), RAMBUS Dynamic Random Access Memory (RDRAM)and/or any other type of random access memory device. The non-volatilememory 816 may be implemented by flash memory and/or any other desiredtype of memory device. Access to the main memory 814, 816 is controlledby a memory controller.

The processor platform 800 of the illustrated example also includes aninterface circuit 820. The interface circuit 820 may be implemented byany type of interface standard, such as an Ethernet interface, auniversal serial bus (USB), and/or a PCI express interface.

One or more input devices 822 are connected to the interface circuit820. The input device(s) 822 permit a user to enter data and commandsinto the processor 812. The input device(s) can be implemented by, forexample, a keyboard, a mouse, a touchscreen, a track-pad, a trackball,isopoint and/or a voice recognition system.

One or more output devices 824 are also connected to the interfacecircuit 820. The output devices 824 can be implemented, for example, bydisplay devices (e.g., a liquid crystal display, a cathode ray tubedisplay (CRT), a printer and/or speakers). The interface circuit 820,thus, typically includes a graphics driver card.

The interface circuit 820 also includes a communication device such as amodem or network interface card to facilitate exchange of data withexternal computers via a network 826 (e.g., an Ethernet connection, adigital subscriber line (DSL), a telephone line, coaxial cable, acellular telephone system, etc.).

The processor platform 800 of the illustrated example also includes oneor more mass storage devices 828 for storing software and data. Examplesof such mass storage devices 828 include floppy disk drives, hard drivedisks, compact disk drives and digital versatile disk (DVD) drives.

Coded instructions 832 (e.g., the machine readable instructions of FIGS.7 and/or 9) may be stored in the mass storage device 828, in thevolatile memory 814, in the non-volatile memory 816, and/or on aremovable storage medium such as a CD or DVD.

Although certain example apparatus, methods, and articles of manufacturehave been disclosed herein, the scope of coverage of this patent is notlimited thereto. On the contrary, this patent covers all apparatus,methods, and articles of manufacture fairly falling within the scope ofthe claims of this patent.

What is claimed is:
 1. A method, comprising: detecting light informationin image data representative of an environment including a primary mediadevice, wherein detecting the light information comprises detecting aglow in the image data corresponding to an expected glow emanating froma secondary media device; measuring a distance between a person and thedetected glow; determining whether the secondary media device is in theenvironment based on the detected light information; and calculating,via a logic circuit, an engagement level for the person with respect tothe primary media device based on the secondary media devicedetermination and the distance.
 2. A method as defined in claim 1,wherein the light information comprises a light pattern, and determiningwhether the secondary media device is in the environment based on thedetected light information comprises comparing the light pattern to asignature.
 3. A method as defined in claim 2, further comprisingidentifying a type of the secondary media device based on the comparisonof the detected light pattern to the signature.
 4. A method as definedin claim 2, further comprising identifying the second secondary mediadevice as a particular product based on the comparison of the detectedlight pattern to the signature.
 5. A method as defined in claim 1,wherein detecting the light information comprises detecting a lightpattern on the person.
 6. A method as defined in claim 1, whereindetecting the light information comprises analyzing a portion of theimage data associated with a detection of a face.
 7. A method as definedin claim 1, further comprising detecting an ambient light condition ofthe environment by sensing an amount of light in the environment, andselecting a signature from a plurality of signatures based on an ambientlight condition of the environment for comparison with the lightinformation.
 8. A method as defined in claim 7, wherein detecting theambient light condition of the environment comprises sensing a firstamount of natural light in the environment and sensing a second amountof artificial light in the environment.
 9. A tangible computer readablestorage medium comprising instructions that, when executed, cause amachine to at least: detect light information in image datarepresentative of an environment including a primary media device,wherein detecting the light information comprises detecting a glow inthe image data corresponding to an expected glow emanating from asecondary media device; measure a distance between a person and thedetected glow; determine whether the secondary media device is in theenvironment based on the detected light information; and calculate anengagement level for a person with respect to the primary media devicebased on the secondary media device determination and the distance. 10.A storage medium as defined in claim 9, wherein the light informationcomprises a light pattern, and the instructions cause the machine todetermine whether the secondary media device is in the environment basedon the detected light information by comparing the light pattern to asignature.
 11. A storage medium as defined in claim 10, wherein theinstructions cause the machine to identify a type of the secondary mediadevice based on the comparison of the detected light pattern to thesignature.
 12. A method storage medium as defined in claim 10, whereinthe instructions cause the machine to identify the second secondarymedia device as a particular product based on the comparison of thedetected light pattern to the signature.
 13. A storage medium as definedin claim 9, wherein the instructions cause the machine to detect thelight information by detecting a light pattern on the person.
 14. Astorage medium as defined in claim 9, wherein the instructions cause themachine to detect the light information by analyzing a portion of theimage data associated with a detection of a face.
 15. A storage mediumas defined in claim 9, wherein the instructions cause the machine todetect an ambient light condition of the environment by sensing anamount of light in the environment, and select a signature from aplurality of signatures based on an ambient light condition of theenvironment for comparison with the light information.
 16. A storagemedium as defined in claim 15, wherein the instructions cause themachine to detect the ambient light condition of the environment bysensing a first amount of natural light in the environment and sensing asecond amount of artificial light in the environment.